Video-based animal behaviour recognition

Project Leaders

Stanley

With the increasing demand for animal products, there has been a growing focus on animal welfare analysis, particularly in the context of social interactions within commercial farms. Disruptive inter-animal behaviors, such as tail-biting and stepping on others, not only impact animal welfare but also reduce the productivity of the farms. Detecting and preventing such behaviors has become a significant concern. While commercial farms have historically used RFID sensors for monitoring, these techniques can be expensive. As a result, researchers have explored alternative solutions, such as video-based animal behavior recognition.

Recent advancements in artificial intelligence, including Multiple Object Tracking (MOT), Joint Detection and Embedding (JDE), and Look Once (YOLO), have demonstrated that video surveillance can provide a low-cost and sustainable solution. The latest technique, MSQNet, has eliminated the need for actor-specific data, unlocking unlimited possibilities. However, further improvements are necessary to enhance the models in the domain of multi-class classification in video-based recognition.

To address this, a novel strategy is proposed. Additional information from the available dataset, such as the Animal Kingdom dataset, can be incorporated into the model from the ground up. By adding an additional class to the dataset, the model can better classify actions and also perform bonus animal class classification, enabling the classification of two different aspects. However, simply adding an additional class may not yield optimal results. Hence, another method called multi-phase transfer learning is introduced, inspired by human learning stages. The model is initially trained only on action annotations using TimesformerInit, and subsequently, the animal class is introduced.

Research has shown that introducing an additional class, in this case, the animal class, and employing multi-phase transfer learning does not significantly improve the precision of action recognition compared to the default model (~74%). However, the precision of animal class classification is better than that of action recognition. When solely incorporating the two classes into the model, the model's performance saturates at around 72%, indicating that the multi-phase transfer approach maintains the original precision while enabling the classification of additional animal behaviors and animal types.

Multi-phase transfer learning allows the model to retain previously learned patterns, and without it, the model's performance would degrade. This approach enhances the model's capabilities in classifying animal behaviors while also considering the type of animal involved.

Overall, these advancements in video-based animal behavior recognition, along with the incorporation of additional classes and multi-phase transfer learning, have the potential to greatly improve the assessment of animal welfare and behavior in commercial farms.

Project Example